Extraction Techniques for Mining Services from Web Sources

The Web has established itself as the dominant medium for doing electronic commerce. Consequently the number of service providers, both large and small, advertising their services on the web continues to proliferate. Such web presences can range from a simple reference to the service provider in a referral page containing many such references to a full-blown web site of the service provider. Creating queriable service directories by mining such web presences will add impetus to e-commerce activities on the web. In this paper we describe new extraction algorithms for mining service directories from web pages. Services are characterized by an ontology consisting of a taxonomy of service concepts, their associated attributes (such as names and addresses), type descriptions for the attributes and attribute identifier functions to locate occurrences of a service’s attributes in a web page. Two central extraction issues that arise in the mining of service directories from web documents are: (i) How to group the attributes related to a service provider entity especially when there are multiple entities in a page as is the case with referral pages? (ii) How to increase the recall by extracting those attribute occurrences that the identifier function fails to locate especially when the domain of the attribute is not completely known or is unbounded. To address the first issue we develop a novel propagation technique for identifying and accumulating all of the attributes related to a service entity in a web page. For increasing the recall we develop a novel unsupervised learning technique that infers the “context” of the attributes in the page and uses the inferred contexts to locate additional attribute occurrences that the identifier functions fail to detect. In contrast to page-specific wrapper techniques the distinguishing aspect of our ontology-directed technique is that it does not rely on any relationship between markup tags and attribute occurrences in the page. Hence our technique is scalable, i.e. it can handle any page relevant to the intended service domains. Our technique is also versatile since it is easy to tune the ontology to cover different types of service domains. We provide experimental results of the effectiveness and versatility of our extraction techniques by mining a database of veterinarian service providers and real estate agencies from web sources. keywords: web mining, information extraction, service directories Contact Author: Saikat Mukherjee E-mail: [email protected]